Deep learning pipeline for statistical quantification of amorphous two-dimensional materials

October 13, 2022 - In recent years, 2D amorphous materials with small bandgaps have been developed. Scientists from the University of Ulm (Germany), Xi'an Shiyou and Xi'an Jiaotong University (China), and Dresden University of Technology (Germany) have developed a deep-learning pipeline to explore the full range of atomic-resolution quantitative data sets of these materials, which could provide new insights into the structural understanding of small bandgap structures.

The discovery of graphene has triggered an explosion in the development of two-dimensional (2D) materials with intriguing properties and extraordinary application potential,1 including transition metal dichalcogenides (TMDs),2 hexagonal boron nitride (h-BN),3 black phosphorus (BP).4 In recent years, their amorphous counterparts, 2D amorphous (a-2D) materials, have experienced a resurgence.5 The combination of atomic thinness and lack of long-range order has given a-2D materials unique properties, such as ultrahigh uniformity, excellent mechanical and chemical stability, and an abundance of catalytically active sites, promoting their applications in flexible electronics, electrocatalysis, and energy storage.5 Consequently, considerable effort has been devoted to the bottom-up synthesis of a-2D materials. For example, single-layer amorphous carbon has been synthesized by laser-assisted chemical vapor deposition (CVD).6 Due to the atomic thinness of a-2D materials, atomic arrangements can be directly observed using aberration-corrected high-resolution transmission electron microscopy (HRTEM)7,8.

Despite the rapid development of TEM instrumentation, obtaining structural information from HRTEM images of a-2D materials is not a trivial task. In contrast to the crystalline form, where the sample structure can be accurately represented simply by a single unit cell and the atoms it contains, the short-range ordering in a-2D materials requires interpretation of the structure as a whole. In other words, each individual atom in the HRTEM image (Figs. 1a and 1b) must be precisely localized. It is worth noting that a 10 nm x 10 nm image of graphene already contains ~ 4000 carbon atoms, making manual labeling extremely time consuming. This severely limits the quantification of amorphousness and prevents the correlation of structural details with synthesis conditions and resulting functionalities. Moreover, the study of reaction dynamics, e.g., the transition from amorphous to crystalline phases in a 2D material, requires the analysis of each frame of the time-resolved image series, leading to an exponential increase in workload. Although dedicated software tools have been developed to automatically determine atomic positions through template matching and 2D Gaussian fitting,9,10 they are not suitable for identifying complex structural features, including disorder and contrast variations, micropores (Fig. 1c), and surface contamination (Fig. 1d).

In addition, defects and dopants have been mapped in untreated and doped graphene and TMDs, enabling dynamic defect tracking in temporally resolved image sequences.11 Since 2D materials are subject to surface impurities and nanoscale fractures, a recent study has addressed the segmentation of undesired structures in graphene images.12 However, previous work has focused exclusively on 2D crystalline materials, while the full potential of neural networks for structural characterization of amorphous materials remains to be explored.

Recently, the formation of local defects in graphene by various stimuli has been studied using CNNs.13,14 For example, Kalinin et al. revealed the structural evolution and dynamics of defect formation in graphene under electron illumination using variational autoencoders.14 Trentino et al. deciphered the defect structures and statistics in graphene after ion bombardment.13

Exploring the potential of neural networks
Inspired by the pioneering work, the scientists have now further explored the potential of neural networks for the structural characterization of a-2D materials. Note that existing CNN analysis has primarily focused on annular dark-field scanning TEM (ADF-STEM) imaging, where the contrast strongly depends on the atomic number and sample thickness. However, in phase-contrast HRTEM imaging, the weak Bragg scattering in a-2D materials results in relatively low contrast (Figure 1a). For example, the contrast difference between monolayer and nonmonolayer regions in HRTEM (Figure 1d) is significantly lower compared to ADF-STEM.15

Nevertheless, HRTEM imaging provides excellent temporal resolution and allows visualization of fast reaction dynamics down to the atomic scale. Using a microscope that corrects for chromatic (Cc) and spherical (Cs) aberration, the scientists have now successfully studied a variety of dynamic processes, including the formation of metal crystal cores in carbon nanotubes15 and the diffusion of Li ions during lithiation and delithiation between graphene bilayers.16 Thus, a robust and versatile deep-learning pipeline that can be reliably implemented on HRTEM datasets despite less than ideal conditions is highly desirable.

In their new study, the researchers developed a U-Net-based fully convolutional neural network (FCN) for comprehensive image analysis in a-2D materials. HRTEM was performed on the Cc/Cs-corrected SALVE microscope operated at 60 and 80 kV. Using an adaptive approach to generate simulated training data, automated atom identification in the experimental images of single-layered amorphous carbon was achieved, enabling subsequent polygon mapping in real space and local and statistical analysis of bond lengths and angles. By coupling multiple networks, we incorporated multitasking capabilities into the workflow, including segmentation of multilayers, impurities , and micropores at different scales, and subtraction of non-uniform illumination intensity. To further investigate the generality of the deep-learning pipeline, the robustness of FCN was demonstrated in various tasks in different sample systems, including identification of defects and dopants in graphene, atomic-scale strain mapping, and statistical analysis of an amorphous polymer membrane.

Compared to conventional neural networks, FCNs can significantly reduce the parameter space, resulting in higher efficiency and throughput.17 Among the various FCN architectures, the U-net has been shown to be a powerful model for image segmentation. It contains a downsampling path and a corresponding upsampling path. By using downsampling steps (implemented by MaxPooling layers), the mesh is able to capture information from different length scales. The network is symmetric with respect to upsampling and downsampling, as each resolution decrease is coupled with a resolution increase. As a result, the output of the mesh has the same size as the input, which is a prerequisite for pixel-wise classification. Also, skip connections between the same resolution filter kernels are introduced to preserve information that may be lost during down-sampling. Residual blocks 18 are used to improve the robustness of FCN to the vanishing gradient problem (i.e., small gradients during training result in only small updates to the weights and biases of the network, slowing down and eventually stopping the training process) by using skip connections to allow gradients to pass through the layers.

Generation of training data
The generation of training data consists of the creation of suitable atomic models and subsequent image simulation. To emulate the short-range order in a-2D carbon, we created a semirandom atomic model using the atomic simulation environment (ASE) (Fig. 2a),19,21 based on which the image simulation was performed (Fig. 2b). Stochastic seed points are placed within a square region using Poisson disk sampling, avoiding seed overlap. Seeds are then subjected to Voronoi tessellation and carbon atoms are placed on the corner points of the Voronoi cell. This procedure provides the labeling of the atomic positions (vertices of the Voronoi cell) and at the same time the labeling of the polygon centers (seed points) (Fig. 2c), which is crucial for the identification of carbon rings with different geometries. The atomic positions and polygon centers are marked by Gaussian peaks with a width of 0.4 Å (Figs. 2d and 3e). In addition, atomic patches are removed from the model to create pores (Figs. 2a and 2f).

A major challenge in a TEM image is the automatic segmentation of impurities, e.g., amorphous 3D carbon, hydrocarbon, and polymer residues from sample preparation. Previous experiments with Bragg filters20 cannot be applied to amorphous materials because there are no sharp diffraction points in the Fast Fourier Transform pattern. Manual labeling for impurity segmentation has been reported, with the disadvantage of low efficiency and lack of flexibility.12

Considering that the goal is to exclude unwanted regions from the analysis, building a comprehensive model for contamination only for a segmentation label exceeds the reasonable effort. However, if the network is tasked with finding monolayer regions instead of contaminated regions, a simple two-layer model might suffice. We created a bilayer model by adding another (amorphous or crystalline) carbon monolayer to the random porous carbon layer (Figs. 2g - 2i). Due to the pores in the random plate, both monolayer and bilayer regions are present in the model. The interlayer spacing is between 1 and 5 Å. AC-HRTEM images are simulated using the atomic models.21 The chosen simulation parameters are comparable to typical experimental conditions (i.e., pixel size, defocus, aberrations, electron dose, etc.). The training data set consists of 5000 simulated images.

Automated image analysis
Figure 3a shows an experimental HRTEM image of a-2D carbon in which surface impurities and micropores are visible. First, FCN determines the positions of carbon atoms and polygon centers throughout the image, regardless of whether a particular region is suitable for evaluation (i.e., a region that does not have a monolayer). Delaunay triangulation is then performed, which establishes nearest neighbor connections. Including the centers in the Delaunay triangulation prevents carbon atoms from connecting across the polygon. At the same time, it allows different types of polygons (e.g., pentagon, hexagon, heptagon, etc.) to be distinguished based on the number of bonds between the center and the nearest atoms. C-C bonds are selected by removing all bonds between atoms and polygon centers, followed by refinement to screen out bond lengths that are physically infeasible (< 0.5 Å or > 3 Å). Next, any atoms or bonds that overlap with regions that were segmented as non-single-layer are deleted, resulting in the output image with superimposed atom positions and polygons (Figs. 3b and 3c). The FCN provides access to a wide range of quantitative data sets (Figs. 3d-3f). First, the order in the close range, i.e., the distribution of bond lengths and angles, can be easily evaluated. The histograms shown in Figs. 3d and 3e were generated by analyzing 29 experimental images with 12 x 10⁵ bond angles and 7.0 x 10⁵ bond lengths in only 48 hours of computation. The precise atomic positions can be converted into pair distribution functions (PDF) that reveal the atomic distances and coordination numbers of the neighbors. Previously, the PDF g(r) has been used by electron diffraction to study the structures of amorphous materials.22 Due to spatial averaging, PDF analysis can be subject to ambiguities in determining structural models.23 For example, the spatial resolution in the diffraction mode is limited by the diameter of the parallel illumination, leading to potential signal detection due to impurities and different phases.24 In other words, the hydrocarbon impurities on the a-2D carbon contribute to the electron diffraction pattern and the resulting PDF. This problem can be easily solved by determining the PDF from the atomic coordinates of monolayer regions. Moreover, the size of the evaluation region can be selected digitally instead of being dictated by the beam size. Apart from generating statistical histograms, direct observation at atomic resolution allows mapping of structural features in real space and leaves no room for ambiguities as they occur in spatial averaging techniques. Since the polygon type was identified during Delaunay triangulation, pseudocoloring was applied to carbon rings of different sizes to decipher their spatial distribution and abundance (Figs. 3c and 3f). Automation of Big Data analysis avoids user bias and prevents overestimation or even misinterpretation of low-frequency events.

Deep Learning Pipeline
Fig. 4a shows 5-7 and 555-777 defects in graphene, and a grain boundary is shown in Fig. 4b. The bond length map allows direct visualization of local strains and thus identification of dopant atoms (Fig. 4c). It is noteworthy that we can track the time-dependent structural evolution of a-2D carbon when we apply Deep Learning to in situ image series.

The scientists adapted the FCN to different materials by feeding the model into the training pipeline. However, this is not always necessary. Taking a shortcut can be appealing, especially when the simulation effort increases significantly due to increasingly complex structure. For example, Figure 4d shows the HRTEM image of an amorphous polyimine thin film immobilized with viologens.25 Ideally, the node and linker molecules form a hexagonal network with a projected unit cell of a = b = 4.5 nm and γ = 120°. However, defective pentagons and heptagons have also been observed. Due to the large lattice spacing, the first-order reflectance is 0.26 nm^-1 . To improve the image contrast of the organic polymer film, a defocus of about 4.5 μm was applied to increase the transmission in the low frequency range. The nodal molecules appear as bright spots in the image, resulting in a high similarity with the a-2D carbon image despite a different scale. Consequently, the network trained on features with similar pixel dimensions could easily handle the image analysis.

In their new study, they automated image analysis of 2D amorphous materials using a deep learning approach. The neural network is able to identify atomic coordinates and segment analyzable regions with high precision. This was achieved by training the U-network solely on simulated images. A semi-random atomic model with porous and bilayer regions led to successful recognition of complex attributes in experimental images, including disorder, micropores, and surface contamination. The robustness of the neural network to intensity inhomogeneity was further improved by introducing background intensity gradients in the training datasets. Implementation on the experimental images of single-layered amorphous carbon and amorphous polymer provided diverse statistical datasets and direct visualization of short-range structures. Although our deep-learning pipeline was developed for amorphous materials, it readily translates to crystalline samples and enables identification of local defects and grain boundaries as well as strain mapping. We anticipate that automated quantification of short-range order will provide new insights into the structural understanding of a-2D systems and lay the foundation for establishing structure-property correlation in this emerging class of materials. The deep-learning approach could also pave the way for an efficient and precise study of dynamic processes, such as the transition from amorphous to crystalline, amorphization, nucleation, and crystal growth.

Resource: Leist, C., He, M., Liu, X., Kaiser, U., & Qi, H. (2022). Deep learning pipeline for statistical quantification of amorphous two-dimensional materials. ACS Nano, 16, 20488-20496.